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Does Time Spent Online have an Influence on Student Performance? 
Evidence for a Large Business Studies Class 


Abstract 

This paper examines, using regression modelling, whether a statistically significant relationship exists between 
the time spent by a student using the course website and the student’s assessment performance for a large 
third year university business forecasting course. We utilise the online tracking system in Blackboard, a web- 
based software system, to extract and calculate the time spent by each student on a range of specific online 
assessment activities across the entire semester. The evidence suggests that time spent online on the course 
website is associated with higher assessment performance. 
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Introduction 

The factors that influence student learning outcomes, particularly in light of recent technological 
advances, have become a topic of immense interest for educators and policy-makers. The rapid 
improvement in internet technology has witnessed a significant increase in the use of web- 
facilitated technology in education worldwide. This increased use has radically transformed 
pedagogical practices, as well as learning and teaching policies, across many universities 
(Eastmond & Ziegahn 1995; Romiszowski 1997; Calafiore & Damianov 2011). It is commonly 
argued that web-facilitated tools (e.g., online testing, discussion boards, emails, i-lectures and 
virtual chat) encourage students to actively participate in online learning activities that are 
expected to improve academic performance (Karayan & Crowe 1997; Smith & Hardaker 2000). 
The empirical evidence for such tools has generally focused on two main areas: (i) comparing 
online with traditional face-to-face course delivery modes (Farinella 2007; Palloff & Pratt 1999, 
2001; Rafaeli & Ravid 1997; Vengroff & Bourbeau 2006) and (ii) an analysis of how variations in 
online-participation measures correlate with variations in student performance within a given 
course (Calafiore & Damianov 2011; Chan, Chow & Cheung 2004; Damianov et al. 2009; 

Michael & Miethe 1989; Zaiane & Luo 2001). The latter have typically examined how students' 
final grades in a course are affected by a single measure of online participation, such as time spent 
on the course website. However, few studies have holistically examined the potentially variable 
influences of different online participation components on student performance outcomes. Further, 
there are no studies, to our knowledge, that have examined whether online participation 
components differently influence various components of student performance (e.g., examinations, 
assignments, etc.). 

From a policy perspective, significant differences in the effectiveness of different online 
participation components on disparate components of student performance has important 
implications for instructors, educators, universities and course website designers. If particular 
online participation components (e.g., discussion boards or forums) are found to have a significant 
relationship with student's performance outcomes, while others (e.g., i-lectures or virtual chat) do 
not, the instructor can divert course-delivery or web-development resources to gain the best 
outcome. Similarly, if there is a differential influence of online-participation components on the 
various components of students’ performance (e.g., assignments or exams), the instructor can 
direct online resources to match the appropriate task at specific times during course delivery. 

This research seeks to answer three questions. First, does online participation on a course website 
(Blackboard), in conjunction with face-to-face learning, improve student learning outcomes in a 
large business-forecasting course at an Australian university? Second, do various components of 
online participation have differential influence on overall student learning outcomes? Third, do the 
various components of online participation influence disparate student performance tasks 
differently? 

There are several motivations for undertaking a study of this nature. First, from a pedagogical 
viewpoint, we seek evidence to test the hypothesis that the provision of online tools (in addition to 
traditional teaching) enhances student performance and learning. This evidence would also be of 
major interest for university administrators who wish to establish whether the significant 
investment in online technology contributes to improved learning outcomes and is cost-effective 
overall. Second, if online participation enhances learning overall, instructors are keen to know the 
pedagogical contribution of each online tool in improving learning outcomes; this knowledge can 
allow instructors to design course websites and use those online tools likely to further engage 
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students in the learning process and improve learning outcomes. Moreover, university 
administrators can allocate investment in online tools in a manner that either optimises learning for 
a given expenditure or minimises expenditure to achieve particular learning outcomes. Third, if 
online tools do, in fact, enhance student performance and learning, it would be instructive to know 
if they are as effective in improving student performance and learning for disparate performance 
tasks. For the instructor, the potential differential effect of online tools on various student 
performance tasks has implications for course website design and allocation of online resources 
over the semester. 

Literature Review 

There has been a great deal of research undertaken on the benefits and costs of online education 
and its impact on student learning. Generally, studies have focused on the characteristics of 
specific online tools, such as online messages, tests and discussion boards, and assessed their 
relative contribution to student learning (see, for example, Palloff & Pratt 1999, 2001; Davidson- 
Sivers, Muilenburg & Tanner 2001; Hara, Bonk & Angeli 2000; McKlin, Harmon, Evans & Jones 
2001; Weisskirch & Milburn 2003; Davies & Graff 2005; Picciano 2002; Nachmias & Segev 
2003; Zaiane 2001; Russell 1999). The empirical evidence typically supports the findings that 
specific online tools can improve and develop student learning. Other researchers have 
investigated the effectiveness of online learning by comparing learning outcomes in courses 
exclusively provided online versus those with a traditional teaching format. In contrast with the 
literature cited above, however, the evidence regarding the effectiveness of online learning tools in 
comparison with traditional teaching is mixed. Harmon and Lambrinos (2007) found that online 
teaching at least did not hinder learning outcomes relative to face-to-face teaching when used in an 
MBA program. Russell (1999) found that there was no significant difference in learning outcomes 
between traditional classrooms and online courses. Farinella (2007) examined the relative 
performance of students and professors in online versus face-to-face teaching and found that the 
cohort of students enrolled in online courses underperformed the traditional cohort by an average 
of 21 points on the final examination. 

Although the evidence regarding the comparative effectiveness of online learning tools is mixed, 
this may be explained, in part, by the different cohorts enrolled in the two types of courses (for 
example, part-time versus full-time, age groups or family responsibilities), the actual courses 
(faculty area, undergraduate versus post-graduate, etc.) or other contingent factors. Therefore, 
unless all important factors are considered and incorporated into the analysis, results and 
conclusions from such studies are problematic and must be treated with caution. However, it 
would appear less problematic to consider the effectiveness of online tools by analysing the impact 
of online usage on student performance within a particular course that uses a hybrid of traditional 
and online approaches. In such circumstances, the impact of many of the factors raised above 
would likely be small, or even negligible. Although fully online courses have been established for 
a number of years, within Australian universities the hybrid-type course, using both face-to-face 
teaching and online tools, is far more common. In general, online tools are used to augment and 
enhance teaching and learning in traditional courses, rather than being employed as pure 
substitutes. Therefore, for the purposes of this study, the evidence within the literature regarding 
the effectiveness of online learning tools when used in conjunction with traditional teaching 
methods would be more relevant and instructive. 

Most of the studies that examine the effectiveness of online tools as enhancements to traditional 
teaching focus on some measure of online participation or engagement, and relate this to measures 
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of student performance. These studies vary not only in the participation measure used in the study 
(such as time spent online, discussion messages read or written, online files downloaded or online 
sessions) but also on the method of data collection. A range of studies have relied on surveys in 
which students self-report the amount of time spent online on study (see, for example, Michael & 
Miethe 1989 and Williams & Clark 2004, among others). The major drawback of these types of 
studies is the validity of the self-reported measures and the lack of completeness of the data. 

Other researchers have tried to overcome the deficiency of self-reporting data by using computer 
logs of course-related online activities (see, for example, Peled & Rashty 1999; Rafaeli & Ravid 
1997; Zaiane & Luo 2001). Rafaeli and Ravid (1997) examined the correlations between student 
achievement and course-related computer-usage behaviour measures and found that students’ 
online participation was able to predict 20% of the variance in the students’ grades. Vengroff and 
Bourbeau (2006) sampled 80 students in two introductory psychology courses and found that 
students who consistently used Blackboard performed better on examinations than students who 
used it less frequently, even after controlling for their grade point average (GPA). 

Recently, several key studies have explored actual time spent online as a performance metric for 
student participation. Damianov et al. (2009), using a multinomial logistic model and a range of 
control variables (GPA, gender, age, major, etc.), found a positive and significant relationship 
between time spent online and grade for a large business course in South Texas, particularly for 
students who obtained grades below B. However, they found no significant difference for students 
obtaining grades from B to A. In a recent study, Calafiore and Damianov (2011) found, using 2008 
data for courses with online components offered by the Economics and Finance Department of a 
large public university in south Texas and an ordered-choice model, that both GPA and actual time 
spent online were associated with higher grades. 

Most of the above studies have used a single measure of online participation and related this to 
student performance in final examinations or final grades, which typically included within- 
semester assessment. However, it is unlikely that a single measure will adequately capture online 
participation, which typically consists of many disparate online activities such as reading files, 
reading and writing discussion messages, participating in virtual chats and downloading and 
listening to iLectures. It is also unlikely that each of these components will have identical effects 
on student performance. The conclusions from these studies must thus extend only to the online 
tool used in the study, not online participation overall. 

Chan, Chow and Cheung (2004) endeavoured to overcome this issue. They constructed a “student 
participation index’’ that included log data on the number of web pages viewed, forum questions 
read and posted and chat sessions participated in and submitted, and found that students with a 
higher index usually achieved better grades. While this index can be viewed as a better measure 
for online participation, the weights used in the construction of the index by the authors were 
arbitrarily set. Any significant differences in weights used in constructing the index would likely 
lead to different analytical results and conclusions. 

Overall, the literature seems to suggest that online tools and online participation enhance student 
learning outcomes. However, most of the studies, apart from Chan, Chow and Cheung (2004) and 
Rafaeli and Ravid (1997), examine the influence of a single online-participation measure on a 
single student-performance measure. It is possible that different online tools (and hence measures 
of online participation) will have different effects on student performance. It is also likely that the 
influence of online tools will vary across various measures of student performance. There is no 
study to our knowledge that simultaneously examines online participation over a range of 
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measures and on more than one student-performance measure. This study, in contrast to previous 
studies, will examine the simultaneous relationship of various online activities, such as discussion 
messages read, online sessions, actual time spent online and online assessments undertaken, with 
two measures of student performance: formal and non-formal examination. In contrast with Chan, 
Chow and Cheung (2004), the weights attributed to the various online tools on student 
performance are not determined arbitrarily, but through regression analysis. The next section 
discusses in more detail the data and methodology used to analyse how various online tools affect 
the two measures of student performance examined in this study. 

Data and Methodology 

We obtained data for this study from a third-year level course in business forecasting at a major 
Australian university. In addition to normal lectures and tutorials that were conducted on a weekly 
basis, a suite of specific materials were made available online via Blackboard. This material 
included lecture slides, tutorial documents and other supplementary materials, such as self-testing 
question banks (with answers), additional forecasting data examples, forecasting notes and a 
discussion board facility, where students would discuss in great detail issues of data management, 
forecasting techniques and other academic content. 

Two dependent variables were considered in this study. Overall student performance (final 
numerical grade) was decomposed into two separate performance measures: their performance in 
the final examination component of the course (weighted at 50% of the overall final numerical 
grade and conducted at the end of the semester) and in the non-final examination components (also 
weighted at 50% of the overall final numerical grade, but conducted within-semester). This 
decomposition of overall student grades was chosen because student learning and performance 
outcomes may be reflected or measured differently by diverse assessment tasks undertaken under a 
range of conditions. 

The non-formal examination component was composed of a number of individual assessment 
tasks (i.e., assignments and class tests) that were conducted within the normal course of the 
semester. There were four individual assessment tasks: assignments 1 and 2 were 10% and 20% 
respectively; class tests 1 and 2 were weighted at 10% each. The assignments were generally 
short-answer questions and forecasting-calculation tasks that were submitted in the form of a 
management-style report. The class tests consisted of 30 multiple-choice and true/false (with 
explanation) questions. Students were not permitted to use notes or aids other than calculators in 
the class tests. The assessment tasks were completed in weeks 5, 8, 12 and 13 of a 13-week 
semester. The marks obtained for each within-semester assessment were weighted (as indicated 
above) and combined to form a total mark out of 100. 

The performance in the final examination was the mark (expressed as a percentage) attained by the 
students in the formal examination for the course, held in the official examination period after the 
end of the 13-week semester. It consisted of questions requiring short written answers and 
calculations. Students were not permitted to use notes or other aids in the formal examination, 
other than calculators. 

The independent variables were derived from two sources: (i) Blackboard and (ii) the university's 
student-data system. The student activity in the various online components was measured by the 
student-tracking facility of Blackboard. Although this did not provide a complete log of student 
activity online, it was possible to decompose total student activity online into relevant 
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components, such as total number of unique sessions, completed online self-tests, number of 
discussion messages read and sent, actual total time spent on course content pages and time spent 
downloading lecture slides and tutorial files. The log of online activity was collected and collated 
immediately after the completion of the final assessment task in week 13. Student characteristics 
such as GPA, gender, country of origin (domestic/international) and any other relevant 
information that could be related to performance were obtained from the student-data system. 

The initial sample size, after accounting for a small number of observations with missing values, 
was 314 students. However, this was reduced to 270 for reasons explained below. The student 
cohort had the following characteristics: 48% were female and 52% male; 8% were Australian 
domestic students and 22% international students; and the students had an average GPA of 2.43 
(max limit GPA = 4). The characteristics of the entire group were broadly consistent with typical 
third-year courses in business and economics at this university. 

The dependent and independent variables are described as follows: 

(a) Dependent Variables: 

(i) Mark_Exam (Yl) - Performance in the formal examination (%). 

(ii) Mark_Other (Y2) - Performance in assessment tasks other than the formal exam (%). 

(b) Independent Variables: 

(i) Sessions: Number of log-on sessions. 

(ii) Total_Time: Total time in minutes spent on activities related to course content (not 
discussion board or online self-testing questions). 

(iii) Disc_Mess: Number of discussion board messages read. 

(iv) Selftest: Number of completed online self-test question sessions. Each online self-test 
session consisted of 20 randomised questions from a large question bank. Students 
were expected to attempt one question at a time and could only submit the test for 
grading and feedback at the completion of the full set of questions. Students were able 
to undertake the self-test assessment as many times as they wished to improve their 
learning outcomes. 

(v) Files: Number of files students accessed related to course content and supplementary 
materials. 

(vi) GPA: Student grade point average; maximum attainable = 4, minimum = 0. 

The quantitative nature of the dependent and independent variables allowed us to conduct OLS 
regressions using SPSS version 19.0. We also used E-Views to cross-check our results. Since the 
measures in this sample represent overall performance for the semester and the cumulative online 
participation over the first 13 weeks, no time-series effects or analysis were considered. 

Discussion of Results 

We undertook exploratory descriptive analysis of the data prior to the regression analysis. The 
distribution plots of all variables were examined for evidence of atypical values. Using statistical 
methods (median, inter-quartile range multiples, etc.), several outliers for each variable were 
identified and the entire observation was removed from the data. This reduced the workable 
sample from 314 to 288 students. In addition, examination of the Disc_Mess data showed a 
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substantial spike at the zero value (18 observations). These observations were removed from the 
dataset due to estimation-bias concerns (censoring), leaving a final data set of 270 observations. 
The distribution plots of the variables were all acceptable, with the dependent variable exhibiting 
approximately normal behaviour. Some of the independent variables exhibited slightly right- 
skewed distributions; however, these were judged not to unduly influence the estimated results. 
Table 1 gives summary descriptive measures for the variables. 

TABLE 1: Descriptive Summary Statistics 


Vars 

Mark_Ex 

MarkjOther 

Sessions 

Total_Time 

DiscJMess 

Selftest 

Files 

GPA 

Mean 

62.87 

68.32 

61.86 

1577.46 

545.82 

3.10 

105.89 

2.42 

Med. 

64.00 

69.15 

57.50 

1350.00 

501.00 

3.00 

98.00 

2.44 

St. Dev 

11.09 

11.05 

25.58 

1053.70 

411.39 

2.26 

38.54 

0.64 

Min 

10.00 

10.36 

12.00 

108.00 

14.00 

0.00 

22.00 

0.67 

Max 

83.00 

91.28 

134.00 

3210.00 

958.00 

9.00 

181.0 

3.83 


Correlation analysis on the variables was also undertaken to assess the strength of association of 
the independent variables with the two dependent variables ( Mark_Exam, MarkjOther) and any 
potential estimation issues arising from collinearity between the independent variables. The 
variables Sessions, Total_Time, Selftest and GPA had reasonably strong correlations with the 
dependent variables, while the independent variables Sessions, Total_Time and Files, as expected, 
exhibited reasonable strength of association. None of the other independent variable correlations 
indicated any potential collinearity problems for estimation. 

Although Sessions, TotalJTime and Files would be expected to be similar indicators of students' 
online participation (outside of self-tests and discussion messages), variation may have occurred 
due to the different length of time each student took in viewing and absorbing online materials. 
Additionally students may have spent their time online on tasks other than opening and 
downloading files, which usually contained lecture slides and additional notes. Therefore, it was 
decided to include all these variables, as well as Selftest and GPA, in separate linear regression 
models for the two dependent variables. Variables such as Gender and Country of Origin 
(domestic/international student) were not considered since the scope of this study was primarily to 
examine the impact of online learning tools on individual performance outcomes. Tables 2a and 2b 
show the estimated results for the regression models using each of the dependent variables 
Mark_Exam and Mark_Other, respectively. 


TABLE 2a: Initial Regression Results - Dependent Variable Y1 (Mark_Excim %) 


Variables 

Unstandardised 

Coefficients 

Std. Error 

Standardised 
Coeff - Beta 

t-values 

P- 

values 

Constant 

33.771 

2.457 


13.744 

0 

Sessions 

0.053 

0.025 

0.123 

2.110 

0.036 

Total_Time 

0 

0.001 

0.027 

0.501 

0.617 

Disc_Mess 

0.001 

0.001 

0.020 

0.388 

0.698 

Selftest 

0.879 

0.237 

0.179 

3.714 

0 
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Files 

GPA 

-0.014 

9.84 

0.015 

0.819 

-0.05 

0.566 

-0.941 

12.017 

0.348 

0 

R 

R-Square 

AdjR- 

Square 

Std. Error 

F 

Sig. 

0.676 

0.457 

0.444 

8.266 

36.840 

0.000 


TABLE 2b: Initial Regression Results - Dependent Variable Y2 (Mark_Other%) 

Variables 

Unstandardised 

Std. Error 

Standardised 

t-values 

P- 


Coefficients 


Coeff - Beta 


values 

Constant 

41.83 

2.537 


16.489 

0 

Sessions 

0.111 

0.026 

0.258 

4.281 

0 

TotalJTime 

0.001 

0.001 

0.083 

1.461 

0.145 

Disc_Mess 

-0.004 

0.001 

-0.14 

-2.604 

0.01 

FinJSelftest 

0.986 

0.244 

0.201 

4.032 

0 

Files 

-0.025 

0.016 

-0.089 

-1.609 

0.109 

GPA 

8.217 

0.845 

0.475 

9.72 

0 

R 

R-Square 

Adj R-Square 

Std. Error 

F 

Sig. 

0.645 

0.416 

0.403 

8.534 

31.267 

0.000 


Overall, the models for both dependent variables are significant (F-statistic) with reasonable R 2 
results for cross-sectional regressions. The diagnostic tests for heteroscedastic errors were 
negative, and further residual tests for non-randomness were all negative for both models. Visual 
inspection of residual plots suggested randomly distributed normal residuals for both models. 

In both regressions the coefficients on the variables GPA, Selftest and Sessions were significant. In 
the regression for Mark_Other (the non-formal examination component) the coefficient on the 
variable Disc_Mess was also significant, while the coefficients on the other variables (Files, 
TotalJTime) were not significant. Additionally, the last three variables were not relevant 
explanatory variables in the regression for Mark_Exam. Surprisingly, Disc_Mess had a negative 
sign in both regressions, indicating increased participation in online discussion board 
communication may decrease student performance and learning outcomes. However, there were a 
number of insignificant variables, indicating that model modification may be necessary before 
such conclusions can be made with confidence. 

There is a likelihood that the insignificance of variable coefficients in these regression models may 
have been due to potential collinearity among key independent variables. In theory, the variables 
Sessions, TotalJTime and Files are alternative measures of the amount of overall online 
participation by students. This is supported, to some extent, by significant estimated inter- 
correlations among these three variables. 
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To mitigate potential collinearity concerns it was decided to represent overall student online 
participation with only two variables - Online_Time and AvgTime_Session - instead of three. The 
Online_Time variable was used as previously defined; AvgTime_Session was determined by 
dividing Online_Time by the overall number of sessions (Sessions). This approach was selected 
for two reasons. First, a range of previous studies had used online time as the measure of online 
participation, and we considered it important for relative comparison of results between those 
studies and ours. Second, the inclusion of AvgTime_Session allowed for the impact of different 
patterns and consistency of time spent in online participation across online sessions to be 
examined. For example, the effect of 100 minutes of online participation on student performance 
over a single session could be differentiated from the same 100 minutes spread over 10 sessions. 
The former pattern may be indicative of a student cramming for assessment tasks in one session, 
while the latter may be indicative of more consistent engagement with course learning materials 
over the semester. Therefore, new OLS regression models were estimated for both dependent 
variables with the significant variables from the initial regressions and the new variable 
representing overall online participation (Tables 3a and 3b). 

TABLE 3a: Final Regression Results - Dependent Variable Y1 ( Mark_Exam %) 


Variables 

Unstandardised 

Coefficients 

Std. Error 

Standardised 
Coeff - Beta 

t-values 

P- 

values 

Constant 

35.068 

2.222 


15.78 

0 

Online Jime 

0.002 

0.001 

0.150 

1.757 

0.08 

AvgTime _Session 

-0.086 

0.057 

-0.120 

-1.498 

0.135 

Disc_Mess 

0.001 

0.001 

0.036 

0.710 

0.478 

Fin_Selftest 

0.927 

0.235 

0.189 

3.942 

0 

GPA 

9.963 

0.816 

0.574 

12.207 

0 


R 

R Square 

Adj R Square 

ltd. Error 

F 

Sig. 

0.672 

0.452 

0.440 

8.286 

43.548 

0.000 


TABLE 3b: Final Regression Results - Dependent Variable Y2 ( Mark_Other %) 

Variables 

Unstandardised 

Std. Error 

Standardised 

t- 

P- 


Coefficients 


Coeff - Beta 

values 

values 

Constant 

45.210 

2.315 


19.527 

0 

Online_Time 

0.004 

0.001 

0.369 

4.143 

0 

AvgTime _Session 

-0.198 

0.060 

-0.278 

-3.318 

0.001 

Disc_Mess 

-0.003 

0.001 

- 0.111 

-2.090 

0.038 

Fin_Selftest 

1.084 

0.245 

0.221 

4.424 

0 

GPA 

8.437 

0.850 

0.488 

9.922 

0 


R 

R Square 

Adj R Square 

ltd. Error 

F 

Sig. 

0.633 

0.401 

0.389 

8.63293 

35.282 

0.000 
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Overall, the F-statistic suggests that the variables in both models were jointly significant. The 
adjusted R 2 values were slightly lower in comparison with the respective initial models, although 
some overall explanatory power was sacrificed to allow for the inclusion of the particular online 
participation variables ( Online_Time , AvgTime_Session ). The diagnostic tests for heteroscedastic 
errors were again negative, as were the residual tests for non-randomness. Visual inspection of 
residual plots suggested randomly distributed normal residuals. The coefficients on the variables 
GPA and Selftest were strongly significant in both regressions, and their respective coefficients 
had the expected signs. In the regression for the formal examination component in Table 3a, 
Disc_Mess had an insignificant coefficient; in contrast, it was significant in the regression for the 
non-formal examination performance (Table 3b). The sign on the coefficient of Disc_Mess in 
Table 3b was unexpectedly negative (and significant), implying that increased viewing of 
discussion-board messages was associated with reduced performance in the non-formal 
examination components of the course. The overall online-participation variable coefficients 
( Online_Time, AvgTime_Session) were both significant, while Online_Time was only significant at 
the 10% level and AvgTime_Session was not significant. The influence of increased online time on 
student performance was a combination of both the Online_Time and AvgTime_Session 
coefficients. The negative sign on AvgTime_Session suggested that spreading a given online time 
across a number of sessions increases the effectiveness of time spent online relative to fewer 
sessions. 

The results have significant implications for developers of online resources that are also used in 
conjunction with face-to-face learning environments. As expected, GPA as a proxy for student 
ability is a major determining factor in student performance. In this study, a one-unit increase in 
GPA - for example, from 2 (Pass average) to 3 (Credit average) - leads to an estimated 
improvement in student performance (for the two assessments) of approximately 8.5 
(MarkjOther ) and 10 ( Mcirk_Exam ) points (the coefficients can be compared between regressions 
since the independent variables are measured identically and both dependent variables are 
measured as percentages). Moreover, each self-test assessment completion suggests an 
improvement to student assessment performance in both assessment types by, on average, one 
point (i.e. number of self-test assessments completed in this sample range from 0 to 9). Although 
the self-testing assessments can be provided off-line, online provision allows the educator to 
internally monitor whether students undertake the assessments and how many they attempt or 
complete, and the time spent on self-testing assessments. Furthermore, online provision provides 
instant marking and feedback for students, which can support learning outcomes. The results are 
consistent with previous literature findings that GPA is an important factor, and that self-testing 
assessments are important in enhancing learning outcomes. 

The negative and significant coefficient on Disc_Mess in the non-formal examination component 
regression (Table 3b) suggests that the reading of discussion messages may have a detrimental 
effect on student performance, although this seems counter-intuitive, as it may reflect the quality 
of the discussion messages, which are almost entirely student-generated. An alternative 
explanation for this negative effect is that increased discussion-board activity may detract from 
other online activities (for example, lecture and tutorial content or self-testing). Furthermore, a 
more sensitive or fundamental measure of discussion-board activity may be required, as a portion 
of communication in discussion boards is typically only peripheral to course content ( such as 
organising group activities or making routine enquiries). However, the negative effect of 
discussion messages read on student assessment performance is not evident for the formal 
examination component. This may be explained by the fact that assessment tasks, such as group 
assignments, have much longer gestation periods and opportunities for students to seek advice 
from other students, instructors and/or other student groups via the discussion board. The evidence 
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regarding the influence of discussion messages is thus rather mixed, and further research in a 
much wider context is needed to provide more-conclusive evidence of whether discussion-board 
activity aids, hampers or has no significant effect on learning outcomes. 

The amount of time the student spends online ( Online_Time ) does seem to affect student 
performance. However, the effect depends on the type of assessment and the number of 
accompanying sessions. The evidence suggests that for the non-formal examination component 
there is a significant effect of time spent online, which appears to be moderated by the number of 
sessions over which the online time is spread. However, the effect of time spent online is only 
marginally significant for the formal examination component, with no evident moderating effect 
from the number of sessions. The evidence from both regressions seems to support previous 
studies' findings that the amount of time spent online on course websites does have some effect on 
student performance and learning outcomes. However, there is some overall evidence from this 
study that the marginal effect of the amount of time spent online is moderated by the number of 
sessions over which the time is spread. 

Overall, there appear to be differential effects from the various online-participation components, 
depending on the type of measure of student performance employed. The influence of GPA as a 
proxy for student ability on performance seems more prominent in the formal examination 
component compared to the other components; however, the marginal impact of the online self¬ 
tests seems approximately the same for both components. The clearest difference between the two 
regressions seems to be in the effect attributed to discussion messages read and amount of time 
spent online. While these two variables are significant in the non-formal examination component, 
they are insignificant or marginal for the examination component. There appears to be evidence 
supporting the notion that the marginal effect of the various online tools varies between student- 
performance measures. 

Conclusions and Implications 

The main objective of this study was to examine whether a variety of online-participation web 
tools in conjunction with face-to-face learning significantly improve student learning outcomes in 
a business-forecasting course. We also endeavoured to establish whether the influence of various 
online-participation web tools varied depending on the type of measure used to assess student 
performance and learning. The overall results seem to suggest that the provision of online web 
tools, in addition to face-to-face learning, improves learning outcomes. Furthermore, online self¬ 
testing assessments that are immediately marked and incorporated with feedback appear to 
significantly increase student performance in assessment tasks. Moreover, increases in the overall 
time spent online (accessing course content, additional examples, additional notes, references, etc.) 
has a positive effect on student performance, although this effect is marginal for the formal- 
examination component. 

The use of the discussion-board facility does not seem to positively affect students' assessment 
performance. In fact, the evidence suggests that it may have a negative effect on learning 
outcomes. While it is insignificant for the formal exam component, it is negative and significant 
for the non-formal examination component, which suggests that reading the discussion board may 
be detrimental to student performance. 

The results of this study suggest that the influence of online components on student performance is 
more complex than previous analysis in the literature has concluded, and different online 
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components may be more suited to particular learning tasks in the overall learning process. This 
implies that previous empirical results are possibly influenced by the measure of student 
performance chosen for evaluation of student learning. For this particular business course, time 
spent online and, in particular the reading of discussion messages, may be better suited to group 
assignments and within semester assessments, rather than learning indicated by final exam 
performance. Further, if these differential effects are borne out by experience and additional 
research, educators, web designers and instructors should adopt different online-participation 
components that best suit different learning tasks. 

Additionally, the marginal effect of time spent online varies depending on the number of sessions 
over which this online time is spread. Spreading the amount of online time over a number of 
sessions may indicate a more consistent and continual engagement with the course. This consistent 
engagement, rather than just the amount of online time alone, may be the reason for improvements 
in student performance and learning. 

For university administrators and instructors, the evidence suggests that, overall, provision of 
online tools enhances student performance and learning. There appears to be a payoff for the 
investment of money (mainly by the university) and time (by the university and the instructor) in 
online technologies, although whether this exceeds the costs of provision cannot be answered by 
this analysis. 

The differential effects of various online tools suggest that modifying course websites' design 
and/or use can enhance student performance. From the above analysis, instructors would be 
advised to continue with self-testing assessments and reduce reliance on discussion boards, which 
seem to negatively affect student performance. In particular, the discussion board’s negative effect 
is greater for the non-formal examination components of assessment. This may suggest that the 
instructor needs to monitor more closely the discussion board during the non-formal examination 
assessments, and possibly filter messages more meticulously during that period. From the 
university administrator's perspective, consistent evidence of the ineffectiveness of particular 
online tools may suggest shifting resources away from these tools to more effective ones. 

The evidence regarding online time and average time per session suggests that it may be better for 
instructors to encourage more-consistent student participation with the course website. This can be 
achieved by instructors drip-feeding materials on a weekly basis (and possibly removing them 
after a certain time), rather than providing them all at once and/or keeping all materials available 
all the time. Alternatively, instructors may provide incentives for students to participate on a 
consistent basis, such as weekly virtual chat sessions, new web links or weekly marks for online 
participation. 

It is important to note that this study, along with other studies of this nature, has a number of 
limitations that provide a basis for further research and investigation. Although the sample size 
was reasonable, a much larger sample across many and varied courses may be required before 
results can be definitively generalised. The course under investigation is a quasi-quantitative unit 
(business forecasting), and the results could be different if the course were more qualitative in 
nature. Furthermore, the course was at third-year level, with more mature and experienced 
students. These results may not be replicated with first-year courses with a younger and less 
experienced cohort of students. 

Further limitations were provided by the measurement of the key variables. Assessment tasks are a 
proxy for student learning, and may not necessarily accurately reflect actual learning. Measures 
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such as logged sessions, total time spent and files accessed may not reflect students’ interaction 
with the online materials. For example, one student may record a logged session to download a 
file, while another may record a session to examine some data examples and read additional notes. 
We are currently in the process of decomposing each student-logged session into more finely 
granulated constituent components (how long downloading files, how long accessing course 
content, etc.) to provide further evidence. 

Similarly, discussion-board activity is proxied by discussion messages read. This may not totally 
reflect students’ engagement with discussion-board communication and its efficacy as a learning 
tool. There is also an argument to suggest that it is the “quality” of the messages rather than the 
volume that influences student learning outcomes. The focus on "higher-quality" messages - those 
judged to contain potential learning content - may provide a better measure of discussion-board 
activity. 

Student classification variables such as Gender and Country of Origin (domestic/international 
student) were not considered for this study. It is possible that the effects of the various online- 
participation components on student performance and learning may be moderated by gender and 
cultural variables that may reflect differences in learning styles. These variables will be examined 
in future studies to gain further insights into the demographic variables that may influence student 
learning outcomes. 

This study is an important step in assessing the usefulness of online tools in improving student 
learning outcomes, given that universities around the world are investing an enormous amount of 
resources in online learning. Overall, the evidence suggests that online materials add additional 
value and enhance student learning outcomes, but the effectiveness of various online tools for 
student learning varies with the type of measure used to evaluate student performance. 
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